Cooling system for contact cooled electronic modules

ABSTRACT

Various embodiments disclose a system and method to provide cooling to electronic components, such as electronic modules or the like. The system includes one or more cold plates that are configured to be thermally coupled to one or more of the electronic components. Internally, each of the cold plates has a cooling fluid flowing inside of at least one passageway. The cooling fluid thus removes heat from the electronic components primarily by conductive heat transfer. An input and an output header are attached to opposite ends of the passageway to allow entry and exit of the cooling fluid. The input and output headers are attached to an external system to circulate the cooling fluid.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of prior application Ser. No.12/339,583, filed 19 Dec. 2008, which is incorporated in its entirety bythis reference.

TECHNICAL FIELD

The present application relates generally to the cooling of compute andstorage systems; and, in a specific exemplary embodiment, to a systemand method of cooling modularly deployed systems without the use offorced air.

BACKGROUND

Enterprise compute and storage systems are increasingly deployed asmodular systems with standardized form factor electronic enclosuremodules mounted in standardized support structures. The standardizedelectronic enclosure modules may be devoted to perform any of a numberof different functions such as computing, storage, or networking. Theenclosure modules are commonly mounted in standardized supportstructures such as 19 inch (approximately 0.482 m) or 24 inch(approximately 0.610 m) wide racks. Such enclosures are commonlyindustry standard 1 U (1.75 inch; approximately 4.45 cm), 2 U (3.5 inch;approximately 8.89 cm), 3 U (5.25 inch; approximately 13.3 cm), or 4 U(7 inch; approximately 17.8 cm) high. Often, the reasons for theadoption of the larger 2 U, 3 U, or 4 U modules is to increasereliability through improved airflow for cooling and to provide spacefor more adapter cards.

Such modular enclosures are customarily air-cooled. They draw air infrom the room they are housed in by means of fans that accelerate theair and force it over the enclosure's internal components to cool them.The resulting heated air is exhausted back into the room. The room airitself is circulated through an air cooler or a Computer Room AirConditioner (CRAC) that is, in turn, cooled by a refrigeration system.Even for moderately powered systems, very large volumes of air must bemoved from the room through the modules, racks, and CRACs. Fans commonlyaccount for 25% of the total power consumed in the modules and racks.CRAC fans consume another 0.1 watt per watt of load. This cooling burdenis passed to the refrigeration system that consumes another 0.3 to 0.4watts per watt of load. The latter load might be increased by hot andcold air mixing in the room, further reducing cooling efficiency. Allthese effects, together with electrical power conversion anddistribution losses, require that, for every watt of power consumed bythe computing section of a server, typically 2.8 watts must be suppliedto a modern best-in-class data center. In many data centers, up to 4watts must be supplied.

In spite of the large amount of energy expended on moving the air, thethermal resistance from the electronic devices internal to a modularelectronic enclosure to the cooling fluid passing through the aircoolers is still excessively high, typically 0.5 degree C./watt to 0.7degree C./watt. This results in a large temperature drop from thedevices to the cooling fluid. For example, a 120 watt processor with apath having a thermal resistance of 0.5 degree C. to the cooling fluidproduces a thermal drop of 60 degree C. In order to maintain a devicecase temperature of 70 degree C., the cooling fluid temperature cannotbe higher than 10 degree C. This requires a refrigeration cycle thatabsorbs considerable energy.

If the thermal resistance could be lowered then the temperature of thecooling fluid could be increased resulting in an improvement of thethermal efficiency of the entire cooling infrastructure. In some cases,the permissible temperature of the cooling fluid could be increasedsufficiently for the refrigeration system to be replaced by a naturalcooling system such as that provided by the evaporation of water in acooling tower or dissipation to groundwater.

Although fluids are sometimes used in cooling electronics, no fullyintegrated, modular, reliable, simple, and cost effective solution hasemerged. Issues to overcome include: difficult installation andmaintenance; modularity and scalability; decreased reliability due tonumerous fluid connections; difficulty in applying the technology toexisting products and environments; and establishing a low thermalimpedance path from the device-to-be-cooled to an external chiller.

BRIEF DESCRIPTION OF THE FIGURES

Various ones of the appended drawings merely illustrate exemplaryembodiments of the present invention and must not be considered aslimiting its scope.

FIG. 1A is a front elevational view of an exemplary cooling framework;

FIG. 1B shows four plan views of exemplary cold plates;

FIG. 1C is a front elevational detail view of an exemplary cold plateand manifold assemblies;

FIG. 1D is a front elevational view of an exemplary assembly of coldplates fabricated as a standalone cold frame;

FIG. 2 is a perspective view of the exemplary cooling framework of FIG.1A mounted in a conventional equipment support structure.

FIG. 3A is a front elevational view of a drawer slide mechanism used forinserting and elevating a module in the support structure.

FIG. 3B is a side elevational view of the drawer slide mechanism of FIG.3A.

FIG. 4A shows a front view of an exemplary embodiment of a cold platemechanism used to engage or disengage a module in a neutral flatposition over the module.

FIG. 4B shows a front view of an exemplary embodiment of a cold platemechanism used to engage or disengage a module showing bending of thecold plate towards the module.

FIG. 4C shows a front view of an exemplary embodiment of a cold platemechanism used to engage or disengage a module showing flattening of thecold plate as the module is pressed against it.

FIG. 4D is a side view of a plurality of exemplary cold plate leverattachments.

FIG. 5A is a front view of an exemplary cold plate in a neutral flatposition over a module.

FIG. 5B is a front view of the exemplary cold plate showing the coldplate bending away from module, thereby releasing the module.

FIG. 5C is a front view of the exemplary cold plate showing the coldplate bending and flattening on the module while simultaneouslythermally engaging the module.

FIG. 5D is a front view of an exemplary cold plate in a neutral flatposition over a module having a different engagement and disengagementmechanism from the mechanism of FIG. 5A.

FIG. 5E is a detail view of an exemplary tube to plate/tube engagementmechanism.

FIG. 6 is a side view of a fluid-filled pouch functioning as anexemplary Thermal Interface Material (TIM) between the module and thecold plate.

FIG. 7 is a side view of a fluid-filled pouch functioning as anexemplary TIM between various module components and the cold plate.

FIG. 8 is a side view utilizing an exemplary flat heat pipe as a secondcold plate.

FIG. 9 is an exemplary embodiment of a TIM placed above a module andconstructed as a sandwich of a thermal pad and thermal fluid.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The description that follows includes illustrative systems, methods, andtechniques that cover various exemplary embodiments defined by thepresent disclosure. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providean understanding of various embodiments of the inventive subject matter.It will be evident, however, to those skilled in the art thatembodiments of the inventive subject matter may be practiced withoutthese specific details. Further, well-known instruction instances,protocols, structures, and techniques have not been shown in detail.

As used herein, the term “or” may be construed in an inclusive orexclusive sense. Similarly, the term “exemplary” may be construed merelyto mean an example of something or an exemplar and not necessarily apreferred means of accomplishing a goal. Additionally, although variousexemplary embodiments discussed below focus on a thermal cooling systemfor electronic components, the embodiments are merely given for clarityin disclosure. Thus, any type of thermal cooling application isconsidered as being within a scope of the present invention.

A fluid cooled system fully integrated with its environment that removesheat from the computer and is directly connected to a remote chillereliminates many of the problems associated with air as the coolingmedium. The energy required to run the fans is eliminated, reducing adata center's energy cost 30% or more. Compute density can be increasedto near physical limits, limited only by the requirements ofaccessibility. A fluid cooled system has little effect on the ambientconditions of its surroundings and is potentially much quieter than anair cooled system. Neither special room configuration modifications norroom cooling are necessary when changing equipment dispositions. Largedata processing systems can be deployed in environments where it waspreviously not possible to do so because of their adverse heat and noiseemissions.

For the purposes of the description of the present disclosure, the term“fluid” includes conventional liquids such as water and phase changerefrigerant fluids that may be in a liquid, a gaseous, or a liquid-gasmixture state. In the simplest case, fluid may be employed to move heatfrom a location hard to cool with air to a place that is easy to cool.

In an exemplary embodiment, a system to provide cooling to electroniccomponents, such as electronic modules or the like, is disclosed. Thesystem includes one or more cold plates that are thermally coupled toone or more of the electronic components. Internally, each of the coldplates has a cooling fluid flowing inside at least one passageway. Thecooling fluid thus removes heat from the electronic components primarilyby conductive heat transfer. An input and an output header is attachedto opposite ends of the passageway to allow entry and exit of thecooling fluid. The input and output headers are attached to an externalsystem to circulate the cooling fluid.

In another exemplary embodiment, a flexible cold plate arrangement isdisclosed that allows electronic modules to be cooled primarily byconductive heat transfer. The flexible cold plate includes a pluralityof tubes adjacently coupled to one another, forming a substantiallyplanar structure. The plurality of tubes are arranged to allow a coolingfluid to flow internally. A first and second manifold is coupled toopposing ends of the plurality of tubes. The first and second manifoldsconnect to a circulation source to provide circulation of the coolingfluid within the plurality of tubes. The flexible cold plate is bentagainst the modules to provide a low thermal resistance.

In another exemplary embodiment, a method of cooling electronicequipment modules is disclosed. The method includes mounting each of theelectronic equipment modules to at least one cold plate formed in asupport structure so as to provide good thermal contact between the twocomponents. The cold plate is connected to an external cooling systemand cooling fluid is circulated between the external cooling system andinternal passageways of the cold plate thus cooling the module primarilyby conductive heat transfer.

In another exemplary embodiment, a method of cooling electronicequipment modules is disclosed. The method includes installing a thinflexible cold plate in proximity to the electronic module. The coldplate has spring-like properties to allow it to bend to present a convexsurface towards the electronic module. The cold plate and the electronicmodule are then brought into thermal contact with one another byprogressively flattening the convex surface against the electronicmodule.

Various embodiments of the present disclosure can make use of ambientair conditioning to maintain the cooling air at a temperature coolenough to cool the case. If many so equipped computers were employed ina data center, a large quantity of air would still have to be movedthrough the local chillers and room environment to maintain an airtemperature that is low enough to cool the hot electronic componentssufficiently. It would be advantageous to have a means to conduct thisheat directly from the case to a fluid means for transport to a remotechiller.

Various embodiments of a cooling structure described herein are designedfor use with, for example, a modular compute or other electronic system.The cooling structure comprises a support structure with cold platesthat may be connected to a conventional data center refrigeration systemthrough a fluid-to-fluid heat exchanger. The support structure may bebased on a conventional 19″ (approximately 0.483 m) equipment rackcommonly used for housing compute servers and other electronicequipment. Such a support structure is adapted to contain a framework ofhollow shelves through which cooling fluid circulates. The hollowshelves act as cold plates to which electronic equipment may bethermally attached for removal of waste heat primarily by thermalconduction.

Upon reading the disclosure given herein, a skilled artisan willrecognize that other types of thermal cooling may occur by, for example,convective or radiative mechanisms as well depending upon the proximityof at least portions of the cold plate to the modular compute or otherelectronic system. The fluid is cooled in the heat exchanger and pumpedto a manifold in the framework. The fluid is then directed to the coldplates via a series of subsidiary pipes and connectors. The fluidabsorbs heat from the modules and exits the cold plates through other ofthe one or more connectors to a collection manifold and then to the heatexchanger.

The support structure has a means of inserting and removing electronicequipment modules (“modules”) into and from the framework and bringingthe modules into thermal contact with one or more cold plates. Noplumbing connections are required to be made to insert or removemodules.

The modules are capable of being cooled primarily by conductive heattransfer to an external cold plate. The modules contain electroniccomponents or subassemblies that thermally contact a side of the modulethat, in turn, contacts the cold plate. Alternatively, one side of themodule may be open with the electronic components or subassemblies indirect thermal contact with the cold plate. As the contacting surfacesbetween the cold plate and module are never perfectly flat or coplanar,and may even be non-rigid and flexible, a compliant thermally conductivesubstance, such as a thermal grease, known independently in the art, oran elastomeric pad (generally referred to as a Thermal InterfaceMaterial (TIM), also known separately and independently in the art), maybe inserted between the contacting surfaces.

The thermal resistance from the cooling fluid in the cold plates to themodules can be less than 2 degree C./W/in.sup.2 (approximately 0.31degree C./W/cm.sup.2) over the thermal interface area between the moduleside and the cold plate fluid. A heat flux of 10 W/in.sup.2(approximately 1.55 W/cm.sup.2) at the module side results in a maximumtemperature rise of 20 degree C. For a module with a well constructedinternal cooling system the module case temperature may be allowed to goas high as 50 degree C. The cooling fluid temperature may therefore beas high as 30 degree C., significantly reducing energy consumption. Withengineering improvements, the fluid-to-module thermal resistance can bereduced to below 0.5 degree C./W/in.sup.2 (approximately 0.078 degreeC./W/cm.sup.2), enabling further energy savings. In such a system, therefrigeration system could be replaced by a natural cooling system suchas that provided by the evaporation of water in a cooling tower ordissipation to groundwater.

Equipment modules may be conventional electronic enclosures such as 1 Ucompute servers, other form-factor enclosures (sometimes referred to aschassis or pods), or unenclosed systems such as server blades or bareserver motherboards. The side of the module to be cooled may be anyside, but is assumed to be the top lid structure in the descriptionherein.

The cooling fluid may be water, water glycol mix, a refrigerant such asR134A, or a variety of other coolant fluids known independently in theart. In the case of a refrigerant, the “cool” fluid entering theframework may be essentially the same temperature as the “hot” fluidexiting the framework, absorbing heat through a phase change rather thana temperature rise.

Framework

With reference now concurrently to FIGS. 1A and 1B, an exemplary coolingframework 100 comprises a plurality of cold plate shelves 101. In aspecific exemplary embodiment, each of the plurality of cold plateshelves 101 is horizontal. However, there is no requirement for thisparticular orientation. Based upon the disclosure given herein, askilled artisan will recognize how to appropriately modify othercomponents as needed for other orientations.

Each of the plurality of cold plate shelves 101 is made up of one ormore flat tubes 102 (FIG. 1B) arranged to be substantially coplanar.Each of the one or more flat tubes 102 comprises a segment of the coldplate shelf. The segments are interconnected and terminated by a firstmanifold pipe 106 (shown as embodiments 106A, 106B, 106C, 106D), and asecond manifold pipe 108 (shown as embodiments 108A, 108B, 108C, 108D)that altogether comprise each cold plate. The one or more flat tubes 102may be of a multi-port rectangular design comprising a plurality ofsmaller tubes attached together adjacently for strength and planaritywhen operating with high coolant pressures.

In an exemplary embodiment, the first manifold pipe 106 is connected toan input header 107 (FIG. 1A) and the second manifold pipe 108 to anoutput header 103 at the other end so as to permit cooling fluidinjected into the input header 107 to flow to the output header 103. Theinput header 107 may further be subdivided into sections 107A, 107B asdescribed below.

The input 107 and output 103 headers may be conventional pipes ormanifolds into which the first 106 and second 108 manifolds may beinserted and welded or otherwise fixed in place to form leak-proofconnections. Each of the input 107 and output 103 headers has an inputconnector 104 and an output connector 105 to pass coolant, respectively,to or from an external cooling system.

In another exemplary embodiment, the input header 107 may be subdividedinto independent sections 107A, 107B such that both the input and outputconnections are made to the same header but through different sections.Fluid will pass from one header section through some of the tubes to theoutput header 103 and be returned to the second section of the inputheader 107 by other tubes. For example, if the input header 107 isdivided into an independent top 107B and bottom 107A section, the inputconnection can be made to the top section 107B. Fluid flows through thetop shelves to the output header 103 where it then travels through thebottom shelves back to the bottom section of the input header section107A. From there, the fluid flows to the output connection 105B to thecooling system. One skilled in the art will recognize that there aremany other possible variations and embodiments of divided headers andflow combinations, including opposite flow directions in different tubeswithin the same shelf. Such variations are intended to be includedherein.

The output header 103 may be eliminated by bending each of the pluralityof cold plate shelves 101 back upon itself making a “U” shaped structurelying sideways, creating two shelves; in such a case, the output header103 is eliminated. Various options and alternative embodiments arereadily imagined by one skilled in the art including serpentinestructures with more than a single bend, thereby reducing the number ofheaders and manifolds required to build a system and reducing the numberof assembly joints.

FIG. 1B illustrates top views of four exemplary embodiments 101A, 101B,101C, 101D of the plurality of cold plate shelves 101. For a firstembodiment 101A of the cold plate, the cold plate segments comprised ofthe one or more flat tubes 102 connect the first 106 and second 108manifolds. The fluid enters a first manifold 106A via the input header107, flows through all the cold plate segments, exiting the secondmanifold 108A at the output header 103. The fluid enters at one cornerof the cold plate and exits the opposite corner to balance fluid flowamong the cold plate segments. A cold plate in the second embodiment101B is similar in construction to the cold plate of the firstembodiment 101A, but is orthogonally mounted.

A cold plate may be constructed such that fluid flows into fewer thanhalf the cold plate segments and then returns through a larger number ofsegments. This allows for expansion room for a refrigerant phase changefrom liquid to gas. In another exemplary embodiment 101C, fluid isdirected into a tube 102B by an entrance manifold 106C. The fluid flowsthrough the tube 102A to a manifold 109 where it is distributed to aplurality of additional tubes 102B and then to an exit manifold 108C.Although each of these cold plate embodiments show one to four platesegments, a skilled artisan will recognize that any number of platesegments may be employed.

In order to accommodate flexure of the cold plates without puttingexcessive strain on the input 107 or output 103 headers, flexible pipes114 are added to the manifolds 106D, 108D of the cold plates in anotherexemplary embodiment 101D. The flexible pipes 114 are made of a flexiblematerial and may include “U,” “S,” or other bend types (not shown) forfurther strain relief.

FIG. 1C is a detail front view cross-sectional drawing illustrating howone of the one or more flat tubes is inserted into the first 106 andsecond 108 manifold pipes.

With reference to a further embodiment shown in FIG. 1D, the output 103and input 107 headers and the first 106 and second 108 manifolds areeliminated. The one or more flat tubes 102 that form the plurality ofcold plate shelves 101 are placed in holes in braced metal boxes 111.This arrangement forms a rigid box with several tiers. The braced metalboxes 111 replace the output 103 and input 107 headers and the first 106and second 108 manifolds. The braced metal boxes in each have an inlet113 and an outlet connection 113 respectively on either side. Oneskilled in the art would understand that other methods of plate supportand interconnection are possible and are thus considered herein.

If the output 103 and input 107 headers are replaced by rectangularboxes, certain areas 110 may have brackets (not shown) affixed to allowthe mounting and cooling of additional auxiliary components orsubsystems. The additional auxiliary components or subsystems caninclude items such as power supplies and network switches that havedifferent form factors from the modules described herein and can alsobenefit from contact cooling.

Other various embodiments not shown, but readily envisioned by a skilledartisan upon reading the disclosure provided herein, include widershelves to accommodate a plurality of modules or rotating the coolingframework such that the shelves are vertical and modules are mountedvertically instead of horizontally. Modules may also be mounted on bothsides of a shelf to halve the number of shelves needed for a givenapplication.

Module Dimensions

For standard 1 U server modules: 1.75 inches (approximately 4.45 cm)high by 19 inches (approximately 0.483 m) wide by 24 inches(approximately 0.610 m) deep, the shelves are placed on, for example, a2 inch (approximately 5.08 cm) pitch. This pitch provides a verticalseparation to accommodate thicknesses of the shelf, the module with aTIM attached to its top, and space to slide it into place withoutdisturbing the TIM. Other module dimensions may be chosen for specificapplications. A person skilled in the art would understand that moduleswith other dimensions could readily be used by adjusting the shelvesaccordingly. Because space for air cooling is not required internal tothe modules, very thin modules may be developed with heightsconsiderably less than 1.75 inches (approximately 4.45 cm), either toenable a standard 1 U module pitch, or to enable much denser compute andstorage systems with less than a 2 inch (approximately 5.08 cm) pitch.

Module Insertion and Support Slides

Referring now to FIG. 2, the exemplary cooling framework 100 is mountedinto a conventional support structure 201. Note that each of theplurality of cold plate shelves 101 can be mounted to and supported bythe equipment rack rather by the headers or braced metal sheets. In sucha case, the conventional support structure becomes an integral part ofthe cold frame.

The support structure 201 is a simple metal frame structure comprisingfour uprights connected together by cross members at the tops andbottoms to form a hollow rectangular box. Drawer-type support slides 202are attached on opposite sides of the support structure 201 between thefront and rear upright members and below each of the plurality of coldplate shelves 101. Modules (not shown) are mounted on the drawer-typesupport slides 202 so that they can be readily slid horizontally in andout of the support structure 201.

The drawer-type support slides 202 may also be used to adjust theelevation of the module so that it fits tightly against the lowersurface of an adjacent one of the plurality of cold plate shelves 101.Each module is slid completely in prior to being elevated to makecontact with the shelf above it. Similarly, the module is lowered priorto removal. This assures smooth operation and eliminates possible damageto the TIM attached to the top side of the module by assuring the coldplates and TIM do not rub against each other during insertion orremoval.

To install a module in the support structure 201, a module is firstmounted on a pair of drawer-type support slides 202 in their fullyextended position, out in front of the support structure 201. The pairof the drawer-type support slides 202 with the attached module are thenslid back into the support structure 201 such that the module nowresides directly below its respective cold plate shelf. The module isthen lifted into its operating position against the cold plate by use ofa lifting mechanism or the plates are brought down against the top ofthe module.

A module lifting mechanism such as the modified drawer slide illustratedin FIGS. 3A and 3B may be employed, however, one skilled in the artwould understand that there are many other mechanisms are possible andare included herein.

FIG. 3A illustrates a frontal cross-sectional view of a modified versionof one of the drawer-type support slides 202 and its attachment to amodule 310. The drawer-type support slide 202 comprises three elements:a support bracket 202A that is typically connected to the front and rearuprights of the support structure 201, a fixed slide rail section 202Bthat is affixed to the support bracket 202A by means of multiplefasteners 306, and a slide rail section 202C that is loosely affixed tothe module 310 by means of multiple pins 305 inserted though exemplaryslits 303, 304 in the slide rail section 202C of FIG. 3B. The multiplepins 305 are firmly affixed to the module 310 but are allowed to slidein the slits 303, 304.

The slits 303, 304 have a profile that define the vertical motion of theattached module 310 as a function of the horizontal motion of the sliderail section 202C relative to the module 310. Slit B lifts the module310 immediately at the beginning of its travel and completes thevertical motion before its travel is complete. Slit A does not startlifting the module 310 until it is partway through its travel. Theseslits are, for example, 4 inches (approximately 10.2 cm) long and eachslit lifts the module 310 0.1 inch (approximately 2.54 mm) in adifferent 2.5 inch (approximately 6.35 cm) section of that travel. Thecombined motions created by these two slits provide incremental contact,first raising the back of the module 310 and then the front of themodule 310, pushing the air out of the space between the TIM on the topof the module 310 and an adjacent one of the plurality of cold plateshelves 101. When the module 310 is removed, the process is reversed,first lowering the front of the module 310 and then the rear. Thismotion, incrementally separating the module 310 from the adjacent one ofthe plurality of cold plate shelves 101 from front to back, helpsovercome any adhesive forces between the module 310 and the cold platewith a minimum of force.

To insert the module 310 in the support structure 201, the slide railsection 202C is first mounted on the module 310 by inserting themultiple pins 305 through the slits 303, 304 and affixing to the module310 such that the multiple pins 305 are in the rightmost positions 305-1of slits 303, 304. At this point, the slide rail section 202C willprotrude out in front of the module 310 by the length of the slits 303,304. Each of the slide rail sections 202C with the attached module 310is then engaged with the respective mating one of the fixed slide railsection 202B and slid fully into it such that the module 310 is fullywithin the support structure 201 and under the cold plate. At thispoint, the slide rail section 202C will still be extended out in frontof the support structure 201 by the length of the slits 303, 304. Theslide rail sections 202C on either side of the module 310 are thenpushed back by handles 301, sliding on the multiple pins 305 to theleftmost position 305-2, thus raising the module 310.

Each of the multiple pins 305 may be any sort of, for example, pin, boltspacer, or screw mechanism that provides a sliding surface whilesecuring the slide rail section 202C to the module 310. Some or all ofthe multiple pins 305 may employ wheeled bearing means or low frictionbushings such as nylon to facilitate a smooth sliding motion.

It can be readily observed by one skilled in the art, upon reading thepresent disclosure, that there are many obvious alternatives to the useof sliders as lifting mechanisms, such as rods with cams or screwmechanisms, that may be used to lever the module into place. Nothing inthis description should be implied to exclude such mechanisms from thisinvention.

A vertical motion to press the module 310 firmly against the cold plateand the amount provided by a fixed mechanism will not always be thesame. Therefore, a spring mechanism or other resilient structure may beprovided to absorb the extra motion and forces exerted when the module310 is lifted into place. There are many methods that can be employedsuch as, for example, metal springs, rubber-like grommets on supportingmembers, flexibility built into the supporting structures, thecompliance of the TIM, or the flexibility of the cooling plate.

FIGS. 4A, 4B, and 4C illustrate various exemplary embodiments to achievecompliance and good thermal contact between the module 310 and adjacentones of the plurality of cold plate shelves 101 by bending the coldplate shelf.

In this series of embodiments, the plurality of cold plate shelves 101are flexible and are individually mounted on the support structure 201via a variety of support and spring mechanisms including a plurality ofhanging brackets 402, levers 403, mounting brackets 404, and spacingwedges 407. In FIG. 4A, the cold plate 101 comprises one or more flattubes 102 fabricated from, for example, thin flat soft aluminum tubesthat are about 0.08 inches (approximately 2.03 mm) thick that are easilyflexed. A thin flexible steel plate 406 about 0.035 inches(approximately 0.889 mm) thick is clamped to the cold plate by means ofthe mounting brackets 404 and fasteners 405. The steel plate 406 acts asa flat spring, resisting bending deformation and providing structuralstrength to the aluminum cold plate. The mounting brackets 404 arefurther mounted on the spacing wedges 407 that are in turn mounted onthe levers 403. The levers 403 are suspended from the hanging brackets402 that are mounted onto the support structure 201. Moving the levers403 apart along the hanging brackets 402 flexes the cold plate away fromthe module 310, while moving the levers 403 closer together flexes thecold plate down towards the module 310.

While a length of the levers 403 can be made less than the pitch of theshelves, it is advantageous to make them longer in order to reduce thehorizontal forces applied to the support structure 201 that are requiredto bend the cold plate. The levers 403 are normally less than the heightof two modules, about 3.5 inches (approximately 8.89 cm) long. Thelevers 403 are moved along the hanging brackets 402 from 0.1 inches(approximately 2.54 mm) to 0.3 inches (approximately 7.62 mm) dependingon the application, rotating about 2 to 6 degrees. This forces the coldplate to bend a nominal 0.1 inches (approximately 2.54 mm) to 0.5 inches(approximately 12.7 mm) down vertically towards the module 310.

The levers 403 on each level are arranged such that levers on adjacentlevels do not interfere with one another. This is accomplished bymounting them at an angle as indicated in FIG. 4D, such that no leverinterferes with the lever above or below it as it is moved along itsrespective hanging bracket 402. A plurality of the hanging brackets 402is attached to the support structure 201 vertically, one above theother. A plurality of connection points 409 attach the levers 403 to themounting brackets 404 outside the vertical line defined by the hangingbrackets 402. As one skilled in the art will readily observe, thehanging brackets 402 may be arranged in other such configurations as “U”shapes to avoid interference by arrangements other than angling. Allsuch lever configurations that avoid interferences are effectivelyunderstood as disclosed herein.

With continued reference to FIGS. 4A-4C, a plurality of various types ofthe module 310 may be envisioned as being suspended below each of theplurality of cold plate shelves 101 on the drawer-type support slides202. A TIM 411 is introduced between the cold plate 101 and the module310 by placing it on top of the module 310. In this position, there is arelatively large space, generally 0.1 inches (approximately 2.54 mm) to0.2 inches (approximately 5.08 mm), between the TIM 411 and the coldplate 101.

When the levers 403 of FIG. 4B are pushed towards one another along thehanging brackets 402, a rotational force is applied, bending the coldplate 101 and the thin flexible steel plate 406 downward towards themodule 310 and its attached TIM 411. The bend forms a convex interfacesurface on the underside of the cold plate 101. When the module 310 isthen raised as shown in FIG. 4C, the TIM 411 is forcibly positionedagainst the cold plate 101. As the module 310 is raised, the now convexcold plate 101 is progressively flattened across the top of the module310, providing distributed pressure over the large TIM/cold-plateinterface, assuring good thermal contact even if the module surface isnot completely planar. The convex cold plate incrementally contacts theTIM 411, eliminating trapped air. Flattening the cold plate by suchpressure pushes the sides out, lengthening it slightly. Thislengthening, from 0.003 inches (approximately 0.076 mm) to 0.03 inches(0.76 mm), is accommodated by compliance of the levers 403. Similarly,when the module 310 is lowered, it also incrementally released,incrementally overcoming any adhesive forces with a minimum of force.

The levers 403 may be permanently fixed in place with the cold plate 101bent into position and contact made by lifting the module 310 intoplace. Alternatively, the module 310 may remain at a fixed height andthe cold plate 101 brought down onto the module 310 by moving the coldplate 101 downward. Another method is to fix the distance between themodule 310 and the cold plate 101, and bend the cold plate 101 with thelevers 403 until the cold plate 101 makes contact with the module.Alternatively, a combination of flexion of the cold plate 101 andvertical movement may be used. To simplify operation, one of a pair ofthe levers 403 may be permanently fixed while only the second of thepair is moved to install or remove the module 310.

One skilled in the art can readily see that the steel plate 406 may bereplaced by another material with suitable flexibility and spring.Likewise, other materials may be substituted for aluminum for the tubes.The separate steel spring may be eliminated by properly tempering thetubes such that they have proper spring-like characteristics.

One skilled in the art, upon reading the present disclosure, willrecognize there are many possible means to construct the hangingbrackets 402 and the levers 403, as well as methods and mechanisms tomove the levers 403. The present disclosure is thus meant to beinclusive of all such means, methods, and mechanisms. These include, butare not limited to, constructing the hanging brackets 402 as screwmechanisms, using cam or sliding lever mechanisms, or affixing thehanging brackets 402 to the levers 403, and moving the hanging brackets402.

FIGS. 5A, 5B, and 5C illustrate other exemplary embodiments to makethermal contact between the cold plate and the module. Entire operatingmechanisms for this series of embodiments are less than 2 inches(approximately 5.08 cm) high, fitting within the height of a singlemodule. As shown in FIG. 5A, an operating mechanism for the module 310resides primarily above the cold plate 101 and alongside a module 310Blocated immediately above the module 310. Standard 1 U modules(stackable with a 1.75 inch (approximately 4.45 cm) vertical pitchwithout a cold plate insert) may therefore be stacked with a verticalpitch of 2 inches (approximately 5.08 cm), or less, including the coldplate mechanism.

Similar to the mechanism described in FIGS. 4A-4C, the module 310 ismounted below the cold plate 101 on the drawer-type support slides 202attached to a support structure (not shown). Affixed to the top of thecold plate 101 is a thin flat steel plate 509 with similarcharacteristics as described above for the thin flexible steel plate406. In this embodiment, the thin flat steel plate 509 extends beyondthe ends of the cold plate 101 and is formed with integrated levers asshown. The levers are U-bends formed on the edges of the thin flat steelplate 509 that are used to control a bending operation of the cold plate101. The thin flat steel plate 509 is riveted through spaces in the coldplate 101 to a plurality of bottom steel bars 505 that extend beyond thefront and back of the cold plate 101, firmly holding the cold plate 101sandwiched between the two steel layers.

Further, the thin flat steel plate 509 may be glued to the cold plate101 to provide extra stiffness. The plurality of bottom steel bars 505have round steel extensions 504 protruding forward and behind the coldplate 101 such that the round steel extensions 504 pass throughhorizontal slots in the support structure. These slots (not shown)support the cold plate 101, permitting a horizontal movement androtation; but limit vertical movement to under 0.01 inches(approximately 0.254 mm). By means of these slots, a space 507 betweenthe underside of the cold plate 101 and the top of TIM 411 can becarefully controlled. The space 507 for this embodiment is between 0.03inches (approximately 0.762 mm) and 0.15 inches (approximately 3.81 mm).

A camshaft 503 of FIG. 5A is fabricated from a round rod 501 ofapproximately 0.25 inch (approximately 6.35 mm) in diameter which servesas the axis of rotation of the camshaft 503. A round pipe 502, about0.75 inches (approximately 19.1 mm) in diameter, mounted off-centeraround the round rod 501, forms the cam. The round rod 501 is mounted inat least two places to the support structure, fixing its location andlimiting its motion to a simple rotation. Dimensions controllinginteroperation of the module 310, the cold plate 101, and the camshaft503 are controlled with a high degree of accuracy by referencing andmounting each of the components on the same support structure (notshown).

Each camshaft 503, in its neutral position, fits snugly under each ofthe U-bends formed at the extensions of the thin flat steel plate 509,extending out from the front and rear of the steel plate. The round pipe502 used to form a portion of the camshaft 503 revolves eccentricallyaround the round rod 501 that functions as the axis shaft. One quarterturn of the camshaft 503 causes the round pipe 502 to press against oneof the walls of the U-bend causing a horizontal motion of between 0.2inches (5.08 mm) and 0.3 inches (7.62 mm) to the right or left,depending on a direction of rotation. The camshaft 503 is turned eithermanually by a handle (not shown) or by a powered mechanism such as anelectric motor (not shown).

The electric motor, if used, can use a reduction gear to give hightorque. The electric motor is mounted on the support structure togetherwith limit switches (not shown). The limit switches constrain thecamshaft 503 to move about one-half turn in either direction. Two motorsmay be used, one on each shaft, or a single motor may be connected toboth through a drive mechanism, such as a chain.

Other possible actuating mechanisms could be driven by hydraulic or airpressure and provide rotational or linear force. Such mechanisms may bereadily designed by one skilled in the art upon reading the materialdisclosed herein. The designs are considered as being disclosed herein.

With continued reference to FIG. 5A of this embodiment, the rotation ofthe camshaft 503 is limited to less than 360 degrees such that thelargest eccentric excursion of the camshaft cannot face upward. Thisreduces the required clearance above the camshaft and thus the totalheight of the mechanism, enabling the 2 inch (approximately 5.08 cm)vertical module pitch to be maintained.

FIG. 5B illustrates a motion of the cold plate 101 when the camshafts503 are rotated to put pressure on outside portions 506 of the U-bendlevers. The plate sides are pushed apart, making the center of the coldplate 101 bow upward away from the module 310, increasing the space 507.This forces the module 310 and the TIM 411 apart, thereby overcoming anyresidual adhesive force between the two components and assuringsufficient vertical clearance to pull out and remove the module 310 fromthe support structure.

The camshaft 503 might also be replaced by a sliding mechanism similarto the drawer-type support slide 202 of FIGS. 3A and 3B. The slidingmechanism would be installed on edge (not shown) as compared to themounting configuration of FIGS. 3A and 3B. In this edge position, slitsor pins on the slide could be used to engage the edge of the steel plate509, moving it side to side in a similar manner as did the camshaft 503.

FIG. 5C illustrates a motion of the cold plate 101 when the camshafts503 are rotated to put pressure on inside portions 508 of the U-bendlevers. Both the compressive forces and rotational torque of thisapplied pressure force the bottom of the cold plate 101 down until itmakes contact with the module 310, eliminating the space 507. If themodule 310 were not present, the middle of the cold plate would dropabout 0.6 inches (15.2 mm). However, with the module 310 present, thecold plate 101 first contacts the TIM 411 in the center as the camshaft503 is turned. This contact area then enlarges as the camshaft 503 isturned further, spreading out from the center, progressing towards eachside until greater than 90% of the TIM 411 surface is contacting thecold plate 101.

As the surface of the thin flat steel plate 509 cannot go down anyfarther than the TIM 411, nor can it be compressed, the forces appliedby the camshaft 503 are absorbed primarily by a spring action of themembers of the U-bend. The temper and spring of the steel plate 509 andthe cold plate 101, the degree of motion imparted by the camshaft 503,the length of the moment arm above the steel plate 509 where thecamshaft 503 and U-bend meet, the overall dimensions of the cold plate101 and the steel plate 509, and the distance from the cold plate 101 tothe module 310, all interact to determine a vertical force appliedbetween the cold plate and module. A minimum pressure of 0.1 PSI(approximately 689 Pa) should be applied to guarantee good thermalcontact between the cold plate 101 and the TIM 411, with a higherpressure desirable. This embodiment can create vertical pressures ofabout 1 PSI (approximately 6.89 kPa), or more, over a module surfacearea of 400 square inches (approximately 0.258 m.sup.2).

FIG. 5D illustrates another embodiment of the construction illustratedin FIG. 5A. Components including the fasteners 405, the round steelextensions 504, and the plurality of bottom steel bars 505 areeliminated. A plurality of rectangular rods 510 is placed above the thinflat steel plate 509, extending into the front and back of the supportstructure (not shown). The support structure thus holds each of theplurality of rectangular rods 510 firmly in place. As the steel plate509 is flexed inward by the camshaft 503 in a first position 508 asshown in FIG. 5C, bending downward onto the module 310 as describedabove with reference to FIG. 5C, the top of the steel plate 509 reactsby trying to rise upward. The steel plate 509 is restrained on each sideby the plurality of rectangular rods 510 thereby exerting downward forceon the module 310, flattening out across the top of module 310 asdescribed earlier.

When the camshaft 503 is rotated outward and upward to a second position506 as shown in FIG. 5B, the steel plate 509 is flexed outward, creatingan upward bow as described earlier with reference to FIG. 5B, and liftedaway from the module 310. In this case, the one or more flat tubes 102of FIGS. 1A-1C need to be attached to the steel plate 509 so as to belifted by it. Note that the one or more flat tubes 102 and the steelplate 509 are bent at different radii and therefore cannot be firmlyattached to each other without casing undo stiffness. At least threedifferent exemplary methods may be employed for lifting the tubes.Although not described in detail, a skilled artisan can readily envisioneach method based upon reading the material disclosed herein.

First (not shown), the tubes can be attached to the steel plate 509 byspot gluing along a mutual center line. Second (not shown), a bracketmay be attached to the top of the tubes such as by gluing or brazingwith the brackets loosely mating to receptors in the plates. Third, thetubes may be bent or embossed upward in a small area and a tab from theplates bent under the tubes, engaging the tubes as shown in FIG. 5E.FIG. 5E illustrates a portion of the steel plate 509 overlying a portionof the tube 102. The plate 509 has a hole 512 cut therein. A tab 513extends from the edge of the hole 512, bending under an embossing 511 inthe plate 509.

With reference now to FIG. 6, an alternative means is illustratedwherein sliding rails are conventionally fixed and the module 310 is notlifted nor the cold plate 101 moved or bent. A small gap between themodule 310 and the associated cold plate 101 above is filled with anexpandable pouch 601 constructed from a thermally conductive materialand having an elongated bulb 604 at one end. A compressible tube 602,filled with air or other gas, is located in the elongated bulb 604. Theremainder of the expandable pouch 601 is substantially filled with athermally conductive fluid.

Prior to the module 310 being slid into the cooling framework, theexpandable pouch 601 is placed either within the support structure underthe associated shelf or directly on the top of the module 310 with therear end of the pouch 601 overlapping the end of the module 310. Duringor after the module 310 being slid into place, the elongated bulb 604 atthe end of the pouch 601 is compressed against a block 605 at the rearof the shelf. The elongated bulb 604 is compressed either by the rear ofthe module 310 or by a lever (not shown) that operates independently ofthe module 310.

The compression forces the thermally conductive fluid to flow into thepouch 601, expanding it and forcing its sides against the cold plate 101and the module 310 filling the small gap. If the gap is filled prior tothe module 310 or lever reaching the end of its travel, the compressibletube 602 within the elongated bulb 604 contracts, thus absorbing excessfluid.

Alternatively, in place of the elongated bulb 604 providing compliance,the block 605 may be designed to offer a needed compliance by othermeans such as a spring (not shown) that limits how much force may beapplied to the fluid in the expandable pouch 601. Other means to forcethe fluid in the pouch 601 between the module 310 and the cold plate101, such as inflating the compressible tube 602, are readilydiscernible to one skilled in the art upon reading the materialdisclosed herein and are thus considered as being within a scope of thepresent disclosure.

In a specific exemplary embodiment, the thermally conductive materialfilling the pouch 601 is an electrically non-conductive and slightlyviscous fluid that will not readily flow out of the pouch 601 should thepouch 601 be pierced or otherwise damaged. The electricallynon-conductive fluid will therefore not damage any electronic equipmentthat any leaking fluid may contact.

When the module 310 is in place, the pressure of the enclosed fluidkeeps the pouch 601 firmly lodged between the module 310 and the coldplate 101. Initiating any movement of the module 310 or releasing thelever that compresses the module 310 will reduce the pressure of thefluid in the pouch 601 making the module 310 easily removable.

Alternatively, the pouch 601 may be used in place of a conventional lidthat may otherwise be attached to the module 310. FIG. 7 shows theexpandable pouch 601 in direct contact with a plurality of electroniccomponent thermal interfaces 702, 704, 712 in the module 310.

Although the description given above generally locates the modules belowthe adjacent cold plates and elevated to contact the cold plates, anyproximate mounting of the module to a cold plate, including above ahorizontal cold plate or alongside a vertically mounted cold plate areconsidered with a scope of the present disclosure. One or more modules(of the same or a plurality of sizes) may also be mounted on oppositesides of the same cold plate. Additionally, the cold plate may be largerthan the module such that multiple modules may be mounted on the samecold plate. Conversely, the cold plate may be smaller than the module.

In another exemplary embodiment shown in FIG. 8, a flat heat pipe 801 isemployed as a secondary cold plate. The flat heat pipe 801 is secured tothe module 310 side such that components and subassemblies internal tothe module 310 are thermally attached to the flat heat pipe 801. Thethermal attachment can occur either by attaching the heat pipe 801 to aside of the module 310, or by using the heat pipe 801 as a lidreplacement in a manner similar to the expandable pouch 601 descriptiongiven with reference to FIG. 7, above. The flat heat pipe 801 includes asection that extends beyond the module side that is moved into thermalcontact with the cold plate 101 when the module 310 is inserted.

The heat pipe 801 may extend straight beyond the module 310 makingcontact to the first cold plate in the same plane as the module side tofrom which heat is extracted. Alternatively, the heat pipe 801 may bendaround a second side of the module 310 as shown in FIG. 8, makingthermal contact with the cold plate 101 mounted orthogonally to thefirst side of the module 310. The bend acts as a flexible spring wherebythe first and second cold plates are thermally attached by pressureexerted by the spring action. A TIM (not shown) may be affixed betweenthe first and second cold plates.

The intersection of the module 310 and surfaces of the cold plate 101may be too irregular to form a good thermal contact even with the abovedescribed means and methods. Conventional TIMs constructed as thermallyconducting sheets of material are either not highly compressible due tothe thermal material fillers, or do not have a high thermal conductivityif they are highly compressible at the applied forces described herein.Thermally conductive grease will flow out of larger spaces and isdifficult to apply and constrain within the prescribed locus ofapplication.

Referring now to FIG. 9, one or more thermally conducting sheetscomprising a first TIM 911 and thermally conducting grease 900 provide ahighly compliant thermal interface. The thermally conducting grease 900is applied in a layer about 0.01 inches (approximately 0.254 mm) thick,more or less depending upon application, but thick enough to readilyflow with applied pressures as described herein, to the top area of themodule 310 that is to be thermally attached to an adjacent cold plate(not shown directly). A frame of about one-half inch (approximately 12.7mm) where no grease is applied is left around the edges of the top areaof the module 310. The first TIM 911, with its one adhesive side down,is placed over the entire area including the frame such that it entirelyseals the thermally conducting grease 900 underneath. When the coldplate is engaged with the module 310, the thermally conducting grease900 will flow from the highly compressed areas into any thermal voids,filling these voids and creating a high quality thermal interface.

In an exemplary embodiment, a viscosity of the thermally conductinggrease 900 is in a range of 20,000 to 200,000 centipoise (20 to 200Newton-sec/m.sup.2). In a specific exemplary embodiment, a viscosity ofthe thermally conducting grease 900 is nominally about 100,000centipoise (100 Newton-sec/m.sup.2).

One skilled in the art will realize the thermal interface as describedherein has applications beyond cooling an external surface of a module.The thermal interface can be used to thermally couple any two surfaces.For example, a component or subassembly internal to the module thatrequires cooling can separately or additionally thermally connected tothe interior side of the module using such a thermal interface.

With continued reference to FIG. 9, an additional set of one or morethermally conducting sheets comprising a second TIM 902, is mounted tothe underside of the module top area over a device 904 requiringcooling. Thermally conducting grease 901 is deposited between the sheetand the module. Further, holes (not shown) may be drilled in the moduletop to thermally couple the thermally conducting greases 900, 901. Theholes allow the grease to flow freely between the top side and undersideof the module, moving to the areas of lesser contact, improving thermalconnection, and conductivity between the cold plate and internalcomponents. Alternatively, the grease could be enclosed in a sealedpouch, such as the expandable pouch 601 described with reference to FIG.6.

Additionally, with reference again to FIG. 1B, any space existingbetween the one or more flat tubes 102 allows room for excess grease toflow, potentially reducing the grease thickness, and thereby the thermalresistance, between the cold plate 101 and the module 310 to a minimum,thereby ensuring good thermal contact.

Although various embodiments have been described herein, it will beevident that various modifications and changes may be made to theseembodiments without departing from the broader spirit and scope ofvarious forms of the present invention. Accordingly, the specificationand drawings are to be regarded in an illustrative rather than arestrictive sense. The accompanying drawings that form a part hereofshow by way of illustration, and not of limitation, specific embodimentsin which the subject matter may be practiced. The embodimentsillustrated are described in sufficient detail to enable those skilledin the art to practice the teachings disclosed herein. Other embodimentsmay be utilized and derived therefrom, such that structural and logicalsubstitutions and changes may be made without departing from the scopeof this disclosure. The Detailed Description, therefore, is not to betaken in a limiting sense, and the scope of various embodiments isdefined only by the appended claims, along with the full range ofequivalents to which such claims are entitled.

Such embodiments of the inventive subject matter may be referred toherein, individually or collectively, by the term “invention” merely forconvenience and without intending to voluntarily limit the scope of thisapplication to any single invention or inventive concept if more thanone is, in fact, disclosed. Thus, although specific embodiments havebeen illustrated and described herein, it should be appreciated that anyarrangement calculated to achieve the same purpose may be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of the various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the above description.

For example, particular embodiments describe various arrangements,dimensions, materials, and topologies of systems. Such arrangements,dimensions, materials, and topologies are provided to enable a skilledartisan to comprehend principles of the present disclosure. Thus, forexample, numerous other materials and arrangements may be readilyutilized and still fall within the scope of the present disclosure.Additionally, a skilled artisan will recognize, however, that additionalembodiments may be determined based upon a reading of the disclosuregiven herein.

1. A flexible cold plate comprising: a plurality of tubes adjacentlycoupled to one another in a substantially planar structure, theplurality of tubes configured to allow a cooling fluid to flow therein;a first manifold to couple a first set of ends of the plurality of tubestogether; a second manifold to couple a second set of ends of theplurality of tubes together, the first and second manifolds beingconfigured to couple to a circulation source to provide circulation ofthe cooling fluid within the plurality of tubes; and a means to bend theflexible cold plate.